Skip to content

Feat/improve monitoring#24

Merged
ClaudiaDuenzinger merged 14 commits intodevfrom
feat/improve-monitoring
Jul 20, 2025
Merged

Feat/improve monitoring#24
ClaudiaDuenzinger merged 14 commits intodevfrom
feat/improve-monitoring

Conversation

@ClaudiaDuenzinger
Copy link
Copy Markdown
Collaborator

Summary
This PR adds documentation and configuration examples for how the Prometheus Operator discovers monitoring targets using ServiceMonitors and Probes in a Kubernetes environment.

Prometheus Operator ServiceMonitor Discovery
The Prometheus Operator manages Prometheus instances using a Kubernetes Custom Resource (CR) called Prometheus.
ServiceMonitors are custom resources that define how Prometheus should discover and scrape metrics endpoints from Kubernetes Services.
The Operator automatically watches for ServiceMonitors in specified namespaces (or all namespaces, if configured).

Discovery process:
The Operator reads the serviceMonitorSelector and serviceMonitorNamespaceSelector fields in the Prometheus CR.
It finds all ServiceMonitors matching the selectors.
For each ServiceMonitor, it matches the selector.matchLabels to Services in the target namespace(s).
For each matching Service, it scrapes the specified port and path (e.g., /metrics or /actuator/prometheus).

Benefits:
Dynamic, label-based discovery of services to monitor.
No need to manually edit Prometheus config for new services.
Supports multi-namespace and multi-team setups.

Prometheus Operator Probe Discovery
Probes are custom resources (if enabled) that define blackbox monitoring checks (e.g., HTTP, TCP, ICMP) using the blackbox exporter.
The Operator watches for Probe resources using probeSelector and probeNamespaceSelector in the Prometheus CR.

Discovery process:
The Operator finds all Probe resources matching the selectors.
For each Probe, it configures Prometheus to scrape the blackbox exporter with the specified module and target(s).
Probes are ideal for monitoring external endpoints, APIs, or HTTP health of services (including databases or exporters).

Benefits:
Enables synthetic monitoring (probes) alongside service scraping.
Centralizes all monitoring configuration in Kubernetes resources.
Key Features Enabled by This PR
Automatic discovery of all ServiceMonitors and Probes in the cluster (with {} selectors).
No need for static scrape configs—all targets are managed via Kubernetes resources.
Supports both application metrics (via ServiceMonitors) and endpoint health checks (via Probes).
RBAC configuration ensures Prometheus can discover targets across namespaces.

@ClaudiaDuenzinger ClaudiaDuenzinger requested a review from sfdamm July 20, 2025 10:57
@sfdamm sfdamm changed the base branch from main to dev July 20, 2025 11:22
Copy link
Copy Markdown
Collaborator

@sfdamm sfdamm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

awesome monitoring dashboards, and automatically discovery. (just added some network changes, and ci/cd pipeline updates).

@ClaudiaDuenzinger ClaudiaDuenzinger merged commit 7d4ab0a into dev Jul 20, 2025
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants